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This report discusses a multiphase project to develop 
a user-oriented survey of prior accomplishments of graduate student 
applicants to aid admissions committees and graduate departmerts in 
the selection process. The three phases included: (1) development of 
three prototype instruments based on a checklist approach, a 
semi -documented approach, and an open-ended portfolio approach; (2) 
development of an instrument with a simplified format that contained 
positive features of the prototypes; and (3) pilot-testing of the 
simplified version, on newly enrolled graduate students, in 26 
university departments (representing fields of English, biology and 
psychology). The third phase investigated the technical soundness and 
the feasibility of the instrument's use in the admissions process. 
Descriptive and correlational analyses of the responses to the 
inventory were conducted to describe characteristics of the students 
and the instrument, and to if'wtify the most reliable clusters of 
items and indices of accomplishment. End-of-the-year follow-ups 
provided student accomplishment data for analyses of the short-term 
correlates of the measure. Faculty were interviewed regarding 
plausibility of the content of the instrument and the utility of 
various reporting formats. Studen* reactions were obtained by 
interview and by additional questions at the end of the inventory. 
The survey is contained in the appendix. (PN) 
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Abstract 



A user-oriented application survey was constructed, 
building on the work of the project to develop an inventory of 
documented accomplishments. The survey was sent to a small 
group of interested departments. The experiences and reactions 
of these departments to th3 structured admissions survey were 
reviewed, and the survey revised accordingly. Eighteen depart- 
ments in three universities agreed to use this revised instrument 
as part of their regular admissions procedures. They were 
subsequently polled for their reactions to the form's utility, 
content, interpretability, advantages over current procedures, 
and comparability H : .th the personal statement. In general, it 
appeared unlikely that a single form could be developed that 
would have appeal and applicability across fields. 
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Foreword 



This report has a long history. Some nine years ago the GRE Board 
Research Committee became interested in the proposition that it should 
be possible to do a better job of assessing significant out-of-class 
accomplishments of applicants to graduate school. While it was 
recognized that the undergraduate grade record and GRE scores have been 
and are likely to remain the principal quantified measures for assessing 
promise for graduate study, few would assert that these measures give a 
complete picture of the candidate. The GRE Board had experimented with 
several types of measures over the years but had not given specific 
attention to assessing personal achievements of students that are 
sometimes quite significant. Students may win prizes, receive 
appointments, publish their work, assume positions of leadership, or 
otherwise gain recognition for unusual competence and pioductivity. 

Such accomplishments would not normally appear in an applicant's 
undergraduate transcript, nor are they assessed in any systematic way. 
The proposition was to develop some type of self-report instrument that 
would summarize and document significant accomplishments. The perceived 
need for such a mechanism stems from the suspicion that such 
accomplishments may not receive the attention they deserve. Certainly 
such accomplishments can be represented in references or by the student 
in filing an application for admission, but does this leave too much to 
chance? The undergraduate grade average and GRE scores have major 
importance in graduate admissions because they are quantified, 
standardized, and highly visible. 

Further, there is concrete evidence that such measures are related 
to success in graduate study. If it were possible to demonstrate that 
significant accomplishments of students could be summarized on a 
standard instrument, and that such accomplishments provide additional 
vrlid information concerning a student's likelihood of success in 
graduate study, then several useful purposes might be served. Among 
these would be an improvement in selection o\ promising students, a 
beneficial broadening of the talent pool, and better means of 
recognizing achievements that deserve reward. Another important 
consideration was the hope that a broader range of demonstrably relevant 
talents might give some applicants (for example, minority students, 
older students) a better chance to put their best foot forward. 

An instrument was developed and it proved quite serviceable for 
research purposes (Baird, 1979). The data gathered indicated that 
significant accomplishments of students do constitute useful additional 
information in appraising candidates for graduate study (Baird and 
Knapp, 1981). In theory, the effort was successful; in practice, there 
were substantial problems. The research instrument proved too detailed 
and cumbersome for routine use in graduate admissions. Among faculty 
there were strong differences of opinion as to the feasibility of any 
such standard instrument. 
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This report gives an account of a limited try with a much 
abbreviated questionnaire developed atd used for experimentally by a 
group of graduate departments. The outcome reported here clearly 
indicates that the single questionnaire could not serve the purpose for 
different departments. An inventory that does justice to the types of 
information of interest to all departments includes much material 
irrelevant to individual departments. The abbreviated questionnaire 
could not overcome that obstacle, and the shortening may also have had 
the effect of appearing to trivialize some types of achievement. 

Another practical prollem may have had an important Influence on the 
way graduate faculty and administrators assessed this inventory. Seme 
departments and schools are very reluctant to make any additional 
demands on applicants. Another instrument may be seen as onerous to 
complete and also run the risk of causing some acceptable applicants to 
view their record and their chances of admission less favorably. 
Departments that are concerned about enrollment may therefore have seen 
such an instrument as working to their disadvantage. 

In setting aside this work for now, the Research Committee has not 
changed its opinion regarding the relevance of an applica.it' s 
significant accomplishments in assessing promise for graduate study. It 
is apparent, however, that this approach to the problem is not likely to 
work. Individual departments might be able to improve their admissions 
process by developing application materials that instruct students more 
specifically as to what type of information is important to their 
candidacy. This report ends with some suggestions that might be useful 
in that regard. 
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Field Trial of a User-Oriented Adaptation of the 
Inventory of Documented Accomplishments as a Tool in 
Graduate Admissions 



Introduction 

A basic purpose of graduate admission procedures is to 
select students who will be likely to be productive, be creative, 
provide leadership, and make a contribution to their fields. 
Graduate schools have always given attention in the selection 
process to indications that students have made significant 
contributions in a field over and above academic qualifications. 
However? many admissions committees and graduate departments 
feel the need for a more systematic way to evaluate the 
accomplishments of students so they can select those who will 
be outstanding in graduate school and will eventually contribute 
most to the field. At the same time, aware of the changes 
iu the nature of the applicant pool, tbey wish to have more 
appropriate selection procedures to evaluate the accomplish- 
ments of oldrr students, students from nontraditional programs, 
and unique and onconventional students. 

The purpose of the project reported here was to develop 
a user-oriented survey of the prior attainments of graduate 
school applicants that would obtain information to meet the 
needs just outlined in a more systematic and organized manner 
than is currently the case and that could be offered for use 
or modification by interested departments. The survey was to 
be based on the instrument in the project to develop 
an inventory of documented accomplishments (Baird, 1979). 

The project to develop an inventory of documented accomplish- 
ments was a multiphase project designed to meet these needs by 
developing procedures to assess the accomplishments of applicants 
to graduate school. In the first phase , trial instruments were 
developed after thoroughly reviewing other attempts at assessing 
accomplishments and carefully considering the issues involved. 
Three prototypes were developed based on three approaches: a 
checklist approach (Holland, 1961), a semi-documented approach 
(Schultz & Skager, 1963), and an open-ended portfolio approach 
similar to that used by the Cooperative for the Assessment of 
Experiential Learning (Knapp, 1975). In the second phase , 
an instrument was developed that was designed to meet the 
operational and conceptual requirements of an inventory of 
documented accomplishments for graduate selection, using as many 
of the positive features of earlier approaches as possible and 
in as simple a format as po&^ible. This version was reviewed by 
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a diverse group of people concerned with graduate admissions for 
the purpose of finding answers to the following questions: 

(1) How open ended should the procedures be? 

(2) How should the quality of accomplishments be evaluated? 

(3) What should be the nature of the content? 

(4) What is the best strategy for documentation? 

(5) What is the best mode of delivery? 

The final version represented the best balance we could 
devise among various answers to these questions. In the third 
phase t the material developed in phase two was pilot-tested 
in 26 departments that represented the fields of English, 
biology, and psychology. The major purpose of this phase was to 
investigate the instrument developed in phase two in terms of: 
(1) technical soundness, and (2) the feasibility of its use in 
the admissions process. 

The departments were asked to administer the instrument 
to their newly enrolled graduate students. Straightforward 
descriptive and correlational analyses ot the responoes to the 
inventory were conducted to: (1) describe characteristics of 
the students and the instrument; and (2) identify the most 
reliable clusters of items and indices of accomplishment. 
Students were followed up at the end of their first year to 
assess their graduate school accomplishments and the relation- 
ship of these accomplishments to the students' previous attain- 
ments. This information provided data for analyses of the 
short-term correlates of the measure. In addition, faculty 
were interviewed to determine the plausibility of the content 
of the instrument and the utility of various reporting formats. 
Students' reactions to the inventory were obtained by interview 
and by additional questions at the end of the inventory. 

The specific goals of phase three were to estimate the 
degree to which information about accomplishments prior to 
graduate school predict graduate rchool success, examine the 
possibility of streamlining the inventory, determine the 
best ways to administer and use the inventory, and examine the 
most useful ways of analyzing and interpreting students 1 
responses and reporting the results. To what extent did this 
phase serve these purposes? The answers to this question, as 
reported by Baird and Knapp (1980), were as follows. 

1. Did the survey assess students 1 significant attainments 
prior to graduate school in a comprehensive, concise, 
accurate and systematic way? The responses to the main 
part of the survey were plausible in the frequency of attain- 
ments, the differences between fields, their intercorrelations, 
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aid the evidence supplied as documentation* Study of the 
character of the responses suggested that almost all students 
completed the survey conscientiously and completely* 
Together y these results suggested that the survey did gain 
information about many significant accomplishments, and that 
the inventory could provide a concise and accurate method 
for assessing pregraduate accomplishments* In addition, the 
f ree-response questions allowed students to present a great 
variety and depth of information about the accomplishments 
they felt were personally significant* Analyses of these 
free-response achievements suggested that two areas of 
significant student activity could be given greater emphasis 
in the inventory: work experience and academic experience 
broadly defined* Thus, it was decided that the inventory 
should ask more questions about these areas* 

Related to the question of comprehensiveness, student 
responses to the evaluation items suggested that some 
students felt that the inventory empnasized quantity and 
tangible products rather than quality and depth, although 
the free-response questions about significant accomplishments 
were designed to allow students to explain the significance 
of their attainments* Others, however, felt that tne inventory 
added to present information* Furthermore, faculty members 
who were interviewed also recognized that the inventory 
collects information systematically and organizes it in 
ways that increase its value for decision making* They 
also recognized that the inventory gave students an equal 
chance to describe themselves and their attainments* Thus, 
in sum, it appears that the inventory did meet this first 
purpose, even though further work could lead to improvements* 

2* Did the inventory identify indicators of broader kinds of 

talent? The statistical results for both the items and four 
experimental scales indicated that they were basically 
unrelated to undergraduate grades* Thus, the inventory did 
provide systematic information about indicators of "non- 
academic" talents that might not appear in the ordinary 
transcript* Furthermore, the evidence provided by a short- 
term prediction study indicated that these indicators were 
correlated with the graduate school attainments of students, 
whereas undergraduate grades were not* Although this is 
undoubtedly partly due to the attenuation in the range of 
grades due to their use in selection, it is worth noring 
that undergraduate grades still correlated .30 with graduate 
grades, which also had relatively little variance* Although 
some of the graduate school attainments were fairly rare 
and the cime span covered only the first year of study, the 
inventory predicted these attainments with moderate success* 
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3. Would the inventory supply information useful in the fair 
evaluation of applicants with special characteristics? 
In the development of the inventory t a strong effort was 
made to incorporate the revisions suggested by reviewers 
from various groups to make the content and phrasing as 
fair as possible. The results of the analyses of both the 
items and scale scores showed some differences between 
men and women students and among students of different 
ethnic backgrounds. However, in each case the groups 
"balanced out." Although men were high in the scientific 
and technical areas, women were high in the artistic and 
social service areas; although Whites were high in science, 
Blacks were high in social service and organizational 
activities. Further, students with different grades and 
students of different ages did not differ significantly on 
any of the scales. Finally, the differences between 
students with different personal characteristics were 
typically much smaller than the differences among the 
fields. This evidence suggested that the inventory provides 
an overall description of students that, taken as a whole, 
allows students with different characteristics to tell 
admission committees what they are good at and what they 
have done. The issue of fairness is very complex, and much 
work would be needed to show that the inventory is unbiased 
in every sense, but these results seem promising. 

The more specific goals of the project also were generally 
met. A careful analysis of students' written responses to both 
the detailed questions and the free-response questions suggested 
that the students' self-reports were accurate. For example, 
when asked to provide documentation, students did provide 
sufficient information to allow a check on their responses. 
Although a complete check of students 1 responses was not 
conducted for reasons of cost, the character of their responses 
suggested that they were responding in good faith and as 
clearly as their understanding of the instructions allowed. 
That is, there seemed to be little exaggeration (e.g., no one 
claimed to have published an article in Atlantic magazine, but 
some said they had published an article in their college 
literary magazine). Although the inventory would have *"o be 
used in actual admissions situations and a study of the verifi- 
cation of students' claims conducted to provide a definitive 
answer, these results suggested that most students responded as 
accurately as they could. 

The results also suggested that the inventory could 
probably be made more efficient and streamlined. First, certain 
achievements were so rare that the items about them could 
probably be eliminated. Further, some items were unrelated to 
any graduate school attainment and could probably be eliminated 



ERLC 



10 



on those grounds. In addition, some activities or products were 
so seldom entered in contests o« sold or so seldom won prizes 
that detailed questions about these points may be unnecessary. 
It seemed likely that a fairly simple and easily completed form 
could be developed. 

The results of the study were less clear as to the most 
appropriate administration and use of the inventory and the 
most useful ways of Interpreting and reporting students' 
responses. Since the study was based on an examination of 
first-semester graduate students rather than actual applicants 
whose responses might be reviewed by admission committees, 
no, hands-on data relevant to these goals were available. 
Consequently, we had to rely on our interviews with students and 
staff. As suggested in the description of the interviews in 
Baird and Knapp (1980), there was little consensus on any of 
these issues. It seemed clear, however, that all the groups 
were favorable to the basic ideas behind the invet.*ory, although 
there was a diversity of opinion about ho* best to implement 
them. Additional work would be needed to work out the best 
conceptual and operational course of action in the future. 

In summary, the basic purposes of this phase appeared to 
have been accomplished. A reasonably comprehensive, concise, 
and accurate method for assessing the pregraduate school 
accomplishments of applicants was developed. The method 
appeared to be fair, at least on initial indications, and to 
correlate with graduate school success, broadly defined. 
Student and staff comments resulted in a variety of suggestions 
for improvements < 

Situation at the Beginning of the Current Project 

At this point we had an instrument that had been subjected 
to numerous reviews by people with very different perspectives. 
It had been shown to organize needed information about graduate 
school applicants' previous accomplishments and to make those 
accomplishments significant ir nhe admissions process. It had 
also been shown to provide information that usefully predicted 
later graduate school accomplishments. However, many questions 
remained about how to make it useful in the day-to-day work of 
the graduate admissions community and to make it helpful to 
applicants. What was the most logical next step to capitalize 
on what we had learned? 

The comments of faculty suggested tbat they had reservations 
about the inventory, in some cases because it was unfamiliar, in 
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some cases because they did not see how it ''ould fit into their 
normal procedures, and in some cases because Zhey felt that it 
overlapped with their current procedures. Students made 
similar comments, particularly that it seemed to duplicate what 
they had reported on application blanks. However, both students 
and faculty recognizr! that it did bring forth information that 
was not often presented on application forms, organized it in 
helpful ways, and increased its significance. One way to 
capitalize on these merits was to change the focus of the 
inventory away from a research instrument to a supplement to 
current procedures by using its content ana format to develop a 
survey that would more directly meet the needs of graduace 
admission committees in their day-to-day work. This would have 
the advantage of using what we had learned to add to familiar 
and accepted tools of graduate admissions. That is, rince we 
have learned what some graduate admissions staff generally want 
to know about applicants, what some applicants want to tell 
admissions staff about themselves, and some of the information 
that is predictive of later graduate school attainments, we i'elt 
it should be possible to restructure the inventory as a tool for 
graduate admissions that: (1) includes appropriate content; (?) 
organizes it in helpful ways; (3) is based on a great deal of 
input from people involved in graduate admissions; and (4) is 
based on an extensive research project. The chief activity 
would be to make user-oriented modifications that would turn the 
research-based results of the inventory project into a form that 
would have wide acceptance. This acceptance would be strongest 
among departments that wish to consider the broad range of 
talent of their applicants, that hope to consider the special 
attainments of unconventional applicants, that would like to do 
so in a systematic way that would be fair to each applicant, and 
that vuint to help each applicant put his or her best foot 
forward* 

How could this be done? The basic goal was tc use the 
content and results of the three phases of the earlier project 
to produce an improved user-oriented version that wcuM have 
broad coverage, based on the results obtained, and would be 
highly acceptable to departments. To do so, the original 
inventory needed to be revised in terms of what was then known 
and in terms of our understanding of what admission committees 
were looking for. Then, to be sur*» that such an admission 
instrument would be usable in regular admissions work, it 
needed to be tried out in a variety of departments to iron out 
any wrinkles of content or approach and to produce a final 
version. 

Specifically, the work of this project was based on phase 
three and consisted of the following steps: 
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1. Streamlining the documented accomplishments inventory by 
eliminating rar<s and unimportant activities and questions 
about details of little significance as they were revealed 
in phase three. The instructions were also simplified. 

2. Increasing the coverage of accomplishments and skills 
applicants may have gained as a result of employment, including 
technical skills, general qualities useful in their fields, 
and interpersonal skills. 

3. Increasing the coverage of academic experiences, such as 
skill in research techniques, organizing ideas, writing, and 
working with other people. 

4. Providing instructions and materials to help students identify, 
describe, and highlight the importance of their experiences 
and accomplishments. 

5. Including a range of fields, such as professional fields, 
that might have somewhat different admissions practices 
(e.g., education, scientific-technical fields, or civil 
engineering) . 

6. Examining how the information is used in actual admission 
decisions (i*e., whether it plays a role and how large a 
role; whether it is particularly useful for certain kinds 
of applicants, such as borderline cases; and admission 
committee members' judgments of its value). 



Developing New Forgg 

The first part of this project consisted of incorporating 
the improvements in the inventory that were suggested by the 
students, faculty, and administrative officers who reviewed the 
inventory and eliminating or changing the items and sections 
that the statistical analyses indicated did not contribute to 
the content or predictive validity of the inventory. Data we 
collected from faculty and student post-invt-itory surveys *s 
well as debriefing interviews from phase three were used for 
instrument .^velopment and refinement. Faculty agreement on 
the content of items and student reaction to the items and 
procedures were the most important criteria to consider. For 
example, some items were psychometrically adequate but had 
questionable meaning or relevance* Some items contributed 
little to the internal consistency of the instrument or the 
item clusters within the instrument and yet were ranked by 
faculty as accomplishments that are significant indicators of 
success in a particular field. Other items were eliminated 
because students and faculty consistently reported them as 
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being troublesome or ambiguous. In brief, there was a reduction 
of items as a result of the analyses of student and faculty 
reactions to the instrument. 

In addition to the analyses of faculty and student reactions, 
the analyses of actual response data were used in several ways. 
Accomplishments that were so rare and esoteric as to be important 
for very few individuals were eliminated. However, a few rare 
accomplishments, such as publishing a scientific article in a 
professional journal, that have obvious and high social or 
potential professional value or that were particularly important 
for a subgroup of students were retained. 

Items that contributed most to the four clusters identified 
in phase three were retained though occasionally in a revised 
form. (The clusters were Literary-Expressive, Artistic, 
Scientific-Technical, and Social Service-Organi national Activity.) 
In addition, the inventory items that had the highest correlations 
with the criteria of first-year graduate school accomplishments 
were retained. 

The coverage ol the inventory was increased in the areas 
of work experience and academic experience. The students* 
descriptions of their most significant attainments suggested 
a variety of attainments in these areas. However, the character 
of their responses suggested that the focus should not be on 
specific accomplishments but on the skills to which the 
experiences led. As noted earlier, these often involved 
technical skills, problem-solving abilities, personal development, 
character traits, communication skills, and interperaotu , 
skills. A format was developed from an examination of the 
students' responses that was intended to aid students in their 
attempts to 'escribe the significance and relevance of their 
attainments. In the effort to expand the coverage of employment 
and academic experience, the advicd of admissions committee 
members and academic advisors was sought. 

Additional revisions led to instructions that focused on 
students' identification of skills gained through their experi- 
ences, questions that led to an explicit description of their 
attainments, and an analysis of the significance of these 
attainments. 

The original form, developed as just described, was sent 
out to approximately 40 institutions. Although some of 
them were willing to use the inventory, the majority were not. 
Telephone interviews with the officials at these institutions 
indicated that they had a number of reservations about the 
inventory. Some *ere concerned abcut the length, some about 
the content, son e about the basic approach. / c ter considering 
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these concerns and reviewing them with researchers at ETS, we 
constructed a new inventory, partly more structured and partly 
more open ended. A brief checklist was constructed from 
the earlier form that was designed to jog the respondents' 
memories and to indicate the sort of specific accomplishments 
we were looking for. In addition, 10 questions were devised 
that a3ked the students to describe their attainments and skills 
in areas such as science, writing, and public service. These 
questions were open ended (see Appendix A) but were designed to 
guide the respondents to be specific about their accomplishments 
and their relationship to their graduate school goals. 

The departments were encouraged to use the application 
inventory as an addition to their normal procedures. The admin- 
istration of the survey was under local control, although 
guidelines for proper and effective use of the admissions 
inventory were sent with the materials. 

It was expected that the inventory would appeal to the 
departments' admission committees partly because it obtained 
the information they often currently collect but in a more 
systematic and orderly way. The content had been reviewed by 
numerous people involved in graduate admissions including the 
Minority Graduate Education Committee of the GRE Board, varlouo 
student groups, and an advisory committee that was specifically 
intended to reflect the diversity of graduate education. In 
addition, tht entire project had been reviewed hy the ETS 
Committee on Prior Review, which is concerned with the privacy 
and quality of data collected in ETS projects. 



Sample and Procedures 

The basic sample consisted of graduate school applicants . 
The students who asked for application forms were sent the 
revised instrument as part of the normal package of application 
materials sent out by the departments. These materials were 
returned to the departments so that the inventory could be used 
in their admission decisions. 

The departments were supplied with all necessary inventory 
materials as well as guidelines, reporting forms, and instructions 
about how to coordinate the work and collect the data in ways 
that were efficient in terms of expense and time. The students' 
replies were returned to the departments. 

Graduate deans in several universities were contacted to 
elicit their support in contacting specific programs. Eventually, 
18 departments in three universities agreed to send out the 
forms with their application materials, and to use them along 
with their usual information in evaluati g the preparation and 
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potential of the applicants. These departments had approximately 
1,200 applicants, although the numbers st specific departments 
ranged from several hundred to less than 10. The forms were 
sent out daring the academic year of 1982-83 • The departments 
were surveyed in spring 1983 to answer six questions. 

1. Generally, how useful did they find the form in admissions? 

2. Was the content appropriate—was it too broad or too narrow? 

3* Did they have any problems in interpreting the responses 
of applicants? 

4. Did they feel that the form was sufficiently useful to 
justify its use on a regular basis? 

5. How would they compare the form to the personal statement? 

6. Did they have any suggestions for imprvoiug the form? 

In addition, telephone interviews were conducted to probe 
the experiences of the departments further. 



Re sults of the Field Trial 

The responses are summarized below around these six topics: 
general utility, content, interpretability, utility beyond 
current information, comparison with the personal statement, 
and suggestions for improvements. 

General Utility . Most participating departments were in 
favor of the general idea of obtaining systematic information 
about the attainments of their applicants, but had concerns 
about the specific mechanism used. The departments recognized 
the importance of including information about the work, extra- 
curricular, and nonacademic accomplishments of applicants in 
admission decisions, and, in general, already had some current 
device to obtain this data. Reactions to the documented 
accomplishments instrument were very mixed ranging from "very 
useful" to "of no value. N Most of these departments would 
probably need to be convinced to use the existing form on a 
regular basis. 

Content . There was little agreement about the content of 
a general purpose instrument. Departments that emphasized the 
performing or plastic arts felt that the questions on the arts 
were not adequate to obtain the information they were concerned 
with, whereas other departments, especially those in the physical 
sciences, considered questions about the arts irrelevant and at 
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least a few spokespersons from these departments considered them 
to be "fun and games," unworthy of being asked of serious students 
in their fields* In general, the attitudes of the respondents 
reflected their fields* Those in the sciences wanted more details 
about their fields* One mathematics department head said he 
was interested in only two things beyond grades and GRE scores- 
had the applicant published an article in a mathematical 
journal and had the applicant worked with a professor the head 
knew or recognized* A department of music was interested only 
in grades and attainments in music* Only a few departments, 
typically those that were interdisciplinary, or those with many 
subfields such as psychology and education, were really 
interested in the broad scope of accomplishments included in 
the inventory* 

Departments with many older students had mixed reactions 
to the form, some indicating that it was too heavily weighted 
toward the traditional younger graduate school applicant, and 
that more information about work experiences should be obtained* 

Utility Beyond Current Information * As might be expected 
from the foregoing, there was a mixed reaction a*z«t the utility 
of the documented accomplishment form* The departments that are 
really interested in the sorts of things assessed by the form 
are already doing something to obtain the information, typically 
using a personal statement or application form specifically 
designed for that purpose* Departments that are not interested 
would not use the form anyway. Thus, those who might use the 
form feel they don't need it, and those who presumably need the 
form wouldn't use it* However, there were a few enthusiastic 
departments that would like to use the form, but in a modified 
version that meets their specific requirements* 

Departments generally felt the form sufficiently supplemented 
their current procedures to warrant its regular use, but with 
qualifications, such as suggesting a form tailored to their 
departments as needed or one that would be appropriate in some 
areas or with some students* 

Interpretability * Interestingly, no department reported 
any problems in interpreting the responses of applicants* 

Comparison with the Personal Statement * The reactions of 
departments were again mixed* Some were quite favorable; for 
example, "First, the systematic approach is useful by jogging 
the person's memory regarding his or her activities* Secondly, 
outside interests (outside of a subject area) are important to 
faculty acceptance judgments, which may not be apparent from the 
personal statement*" However, other departments were very 
satisfied with their current personal statements* 
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Suggestions for Improvements . There were a variety of 
suggestions for improvements, many of which have been noted in 
the previous pages, such as including more on employment 
experienc more questions in the department's area of concern, 
and the like. One department suggested that the form be 
presented in a less formal and threatening format. 

Discussion 

What are the implications of these findings for t'..e 
assessment of the skills and attainments of graduate school 
applicants? First, the disagreement about content suggests 
that there is no way to have a single form that would satisfy 
every department In every discipline. Given the vehemence with 
which some departmental spokespersons expressed their views, it 
even seems unlikely that different forms for broad areas of 
study, such as the physical sciences, the performing arts, 
and so on, would be acceptable, or that even discipline-specific 
forms would be widely used. For example, consider psychology: 
some departments are primarily experimental in their orientation, 
some are clinical, some are social. Even the departments that 
share a common orientation look for different things. For 
instance, a clinical program might emphasize individual therapy, 
community involvement, or research. In addition, many departments 
are generally satisifed with the personal statement and those 
that do not use the personal statement still look for certain 
attainments among their applicants that could be assessed by 
something like the personal statement. 

All of this suggests that the only documented accomplishment 
forms that would be used by most of the departments in the 
study would be forms individually designed to meet their particular 
interests. Rather than attempt to develop broad scale inventories, 
then, a more reasonable strategy may be to develop guidelines 
for local development of accomplishment forms. 

A tailored documented accomplishments form would represent, 
in a sense, a focused, structured personal statement. It would 
obtain information about the individual's qualities assessed in 
the personal statement but would do so in a systematic way. 

Although it would be difficult to outline the content of 
such a guide at this point, it almost certainly would include 
elements such as the following: 
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— Defining the purpose of the survey; for example, whether it 
is to be used as a device to obtain information about the 
accomplishments &nd skills of all applicants, or whether 
its use would be limited to borderline cases. 

— Ob' aining the ideas o£ people in the department about 
appropriate content, establishing legitimacy for the form, 
and explaining its role as providing information beyond the 
transcript. 

— Defining the skills or qualities about which information is 
desired. The skills identified as important by departments 
and students in this project and phase three of the documented 
accomplishments project would be a beginning point for 
this endeavor. 

— Defining or locating "marker" accomplishments that have 
defined skills among previously successful students. 
Again, the accomplishments found to be important in this 
study and phase three would be useful. 

— Writing instructions and questions for applicants. This 
might include explaining the admission procedures used by 
the department, describing kinds of students the department 
is looking for, and outlining the reasons the department 
is looking for such students. The instructions might also 
emphasize that the accomplishments survey is intended to 
provide applicants with an opportunity to tell the department 
about themselves and their special qualities. 



This general approach seems, on the basis of the current 
project, to be the most sensible, and any further work should 
piobably be along these lines. 
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Appendix A 
Application Survey 



A-1 



A-2 



Dear Applicant: 



In addition to your usual academic credentials, we are interested in your experiences 
and accomplishments beyond those that might appear on your transcripts. These may 
have occurred through work experience, special academic work, community service, work 
in organizations, etc. This foru is intended to provide you with an opportunity to 
list and explain the significance of your experiences and accomplishments. There are 
two pari*. The first is simply a checklist designed to jog your memory about things 
you may have done. The second asks for more detail about the kinds of experiences 
that have been found to br .ant for graduate work. Research has shown that 

these kinds of accomplishm predict accomplishment in graduate school, so we are 
interested in what you have done. 

I. Please circle the number of the statements that describe something you have done. 
Please don't be discouraged by this list. Only an unusual student will have had very 
many of these experiences or accomplishments, 
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1. Built scientific equipment (labora- 
tory apparatus, a computer, etc.) 
on my own (not as part of a course) 

2. Was appointed a teaching or 
research assistant in a scientific 
field 

3. Received a prize or award for a 
scientific paper or project 

4. Gave an original paper at a conven- 
tion or meeting sponsored by a 
scientific society or association 

5. On my own (not as part of a course) 
carried out or repeated one or more 
scientific experiments, recorded 
scientific observations of things 
or events in the natural setting, 
or assembled and maintained a 
collection of scientific specimens 

6. Author or co-author of scientific 
or scholarly paper published (or in 
press) in a scientific journal 

7. Invented a patentable device 

8. Member of honorary scientific 
society 

9. Entered a scientific competition 
of any kind 

10. Wrote an unpublished scientific 
paper (not a course assignment) 

11. Served as a research or laboratory 
assistant either in college or 
outside of college 



12. Active member of student groups 

13. Elected as a member of a campus-vide 
student governing body, such as 
student council, student aerate, etc. 

14. Elected as an officer of a campus- 
wide student governing body or 
community group, such as student 
senate, charity volunteers, etc. 

15. Appointed to one or more offices in an 
' organization 

16. Elected president of a class (fresh- 
man, sophomore, etc.) in any year of 
college 

17. Elected as one of the officers — other 
than president— of a class in any 
year of college 

18. Served on a student-f acul y committee 

19. Elected as an officer of a student 
social group or housing unit such as 
sorority, dormitory, pep club, etc. 

20. Elected president of a "special 
interest" student club, such as 
psychology club, mountain climbing 
club, etc. 

21. Received an award or special recogni- 
tion of any kind for leadership 

22. Held J or responsibility for other 
persons (e.g., custodial care, 
emergency squad) 

23. Supervised group of volunteers 
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24. Raised or managed money for 
organization or project 

25. Organized a college political 
group or campaign 

26. Worked actively in an off-campus 
political campaign 

27. Won a prize, award, or other special 
recosnition for scholarship 

28. Had poems, stories, essays, or 
articles published in a public (not 
college) newspaper, anthology, etc. 

29. Wrote one or more plays (including 
radio or TV plays) which were 
given public performance 

30. Acted in a play 

31. Held lead in play 

32. Performed music publicly 

33. Showed art work publicly 

34. Won prize or recognition in music 

35. Won prize or recognition for 
art work 

36. Was feature writei , reporter, etc. 
for college paper, annual, magazine, 
anthology, etc. 

37. Was editor for college paper, 
annual, magazine, anthology, etc. 

38. Did news or feature writing for 
public (not college) newspaper 

39. Had poems, stories, essays, or 
articles published in a college 
publication 

40. Wrote an original, but unpublished, 
piece of creative writing on my 
own (not as part of a course) 

41- Won a literary prize or award for 
creative writing 



42. Systematically recorded my observa- 
tions and thoughts in a diary or 
journal as resource material for 
writing 

43. Member of student honorary group 
in creative writing or journalism 

44. Held job that taught a skill important 
for graduate work in field 

45. Received a job promotion for 
outstanding performance 

46. Started own business 

47. Received award or formal recognition 
for outstanding accomplishment in 
any field 
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A-4 



II. This section describes various attainments which you may have. It is unlikely 
that you will have done something significant in every category, so read over 
the descriptions, and choose the ones that are particularly important to you. 
Then please provide details about your significant experiences and accomplish-* 
ments following the instructions In each section. 



1. Describe any scientific or scholarly writing you have done, its nature 

(class assignment, independent research collaboration with a faculty member, 
etc.), and the extent of its publication (never published, submitted for 
publication, published locally, regionally or nationally) • Also describe 
the type of publication or journal in which it may have appeared. 



2. Please describe any original literary or popular writing you have done 
(e.g., fiction, non-fiction, poems, plays); the circumstances of their 
production (class assignment, independent production, commissioned work, 
etc.); and any publication that may have resulted. Please describe the 
circulation (local, regional, national) and the nature of any publication 
(literary magazine, special interest magazine, popular magazine, college 
newspaper, etc.). 
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A-5 



3, Please describe any literary or scientific contest in which you may have 
participated, the nature of your participation (entrant, judge, etc.)* 
the area from which participants were drawn (local, state, regional, 
national), and any awards or recognition you may have received. 



4. Please describe your role in any scientific project in which you may have 
participated (director, designer, technician, field worker, lab assistant, 
etc.), what the project was designed to accomplish, and its character 
(original experiment, replication of other research, part of research 
program in a job, etc.). Please describe specifically what you did (for 
example, computer programming, interviewing, statistical analyses, building 
equipment, operating equipment, analyzing specimens, etc.), and what the 
outcomes were (project report, journal article, etc.). 
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5. Describe the most important meetings of scholarly or scientific groups you 
have attended, their nature (local, state, regional, or national) and your 
role in these meetings (attended, discussant, secondary author of a paper, 
or first author of a paper). 



6. Please describe any teaching experience you may have. Whom did you teach 
(primary school students, high school students, college students, adults, 
etc.), what did you teach, and for how long did you teach? Did you 
receive any formal recognition of your teaching (certificate, formal 
commendation, etc.)? 
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A-7 



7« Please describe any community involvement you may have had. Please 
describe what your role was (organize?, member, service worker, etc.), 
the level of your responsibility (managed budget, served as vice president, 
et".)» the impact or results of your activities for the people you served, 
and the skills you believe you learned or exercised through you*; involvement. 



8. Please describe any technical skills you may have (e.g., computer programming, 
intelligence testing, editing, spectographic analysis, proficiency in a 
fcreign language, statutes, etc.). Describe where you learned them 
(course work, employment, etc.), and your estimate of your level of skill. 
Please note any forma. 1 certification of your level of skill (course 
completion, certificate, award, etc.). 
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A-8 



9. If you have exercised considerable responsibility or leadership in some 
group or setting (club, employment, politics, etc.), please describe your 
role or position, the size and nature of the group and the level and 
character of the responsibility or leadership. 



10. Please describe any other significant accomplishments or experiences that 
you feel shew your involvement, independence, self-discipline, ambition, 
persistence, responsibility, or interest in your field. Also explain 
the pertinence of these accomplishments or experiences to your graduate 
school goals. 
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